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We propose a methodology for studying the processes of learning and teaching 
mathematics. We will argue that this methodology carries many of the advantages of 
qualitative approaches while still keeping the main advantage of quantitative 
research methodology, which is the potential to produce findings that could be 
generalized. At the heart of this methodology is a research tool that quantifies 
learning processes and enables visualization of the learning paths. This is achieved 
by means of a comprehensive graphical profile of the processes. This graphical 
profile of an individual learner lends itself to detailed analysis and offers new kinds 
of insights into the learning/teaching processes. It is unique in the kind of conclusions 
it suggests and in the scope of the conclusions it allows one to make 

INTRODUCTION 

The wish to understand how students construct mathematical concepts led us to 
develop a methodology that focuses on processes rather than on their outcomes. 
Since the essence of Mathematics is abstractions and abstract concepts, we tested this 
methodology in an exploratory research study of abstraction processes that are 
involved in the learning of a piece of abstract mathematics. We present this 
methodology vis-a-vis our exploratory study. 

We begin our presentation with an overview of our research and then we get into a 
more detailed description of: a) The subjects; b) What concept we used in the 
learning experiment and why; c) What processes we observed and how we initiated 
them; d) How we conveyed the learning text and why; e) The assessment attribute of 
our tool or, How we monitored the processes; f) The products of our tool - The 
individuals learning profiles; and, g) The analytical attributes of the method - Our 
research conclusions. 

The main purpose of this paper is to present our methodology, rather than to present 
the outcomes of our study. As our study was of an explorative nature, we do not 
claim validity for our findings, rather we consider them as conjectures to be studied 
further. 

CONCEPTUAL FRAMEWORK 

Here we look briefly at a part of the clinical context of our study. A laboratory 
experiment seems a reasonable choice given that the goal of the study is to 
understand the basic learning mechanisms themselves, and that “learning takes place 
inside the learner and only inside the learner” (Simon, 2001, p.2 1 0). It might be said 
that this stands in opposition to the contemporary view of learning as a social 
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process; however, Simon’s (2001) perspective of the social aspect of learning 
suggests otherwise: “...‘Social’ means more than ‘having people around’. ... Perhaps 
the most important social influence on learning are ... forms of written 
communication ... Putting learning in books does not desocialize it.”(p. 207). 
Elaborating on this line of thought, we assert that laboratories do indeed incorporate 
the social values and beliefs of the ‘social construct’ that built it. Thus, conducting 
research on learning in a lab environment does not in itself oppose the current social 
trend in the educational milieu; rather, it offers another level of observation. 

Some might argue that from a pedagogical point of view, a 'clinical' context for a 
learning experiment is not appropriate, as their main interest is in learning as a social 
activity (usually in a classroom context). Atkinson et al (2000, p.185), however, 
argue for the transferability of lab experiments to the classroom context and are 
strongly in favor of labs as a context of research on learning. 

One-way to partly control the effect of the teaching method on the learning process is 
to present the concepts to be learned only by examples of the concept. Atkinson at al 
(2000) present a comprehensive literature review of the Leaming-From-Examples 
(LFE) research paradigm. They refer to learning from ‘worked’ examples, which 
provide an expert problem-solving model for the learner to study and emulate 
(p. 181). In most research on problem-solving, the LFE serves as the focus or as the 
target of the research, and not as its context. This kind of research aims at gaining 
insight into the LFE as a (preferred) mode of teaching or of learning, and the subject 
matter serves as the context. Flowever, researchers of concepts-formations (1950- 
1970) used LFE in their experiments in a different way; they used examples as 
instances of a concept. Atkinson et al (2000) put it thus: “A typical study ... 
measured students’ ability to identify a member of a target concept after viewing 
numerous instances and non-instances of it, to ... derive the underlying concept 
common to the examples”, (p. 1 82). In concept-formation research the LFE paradigm 
serves only as a tool to convey the concept to be learned. As such, the LFE is part of 
the context of the research, rather than its focus. 

OVERVIEW OF THE RESEARCH 

We closely observed two abstraction processes that we had identified theoretically: 
Abstraction Process (AP) and Reversed Abstraction Process (RAP). For the purposes 
of this paper we define the two learning processes by the mode of their instigation. 
One is instigated by learning from a text which is presented in an order of increasing 
abstractness level (AP - group U in the experiment - ten ninth-grade subjects), while 
the other is instigated by learning from a text which is presented in an order of 
decreasing abstractness level (RAP - Group D in the experiment - ten ninth-grade 
subjects). Computerized learning modules presented the learning texts, which were 
identical in both modules except for the order in which the chapters appeared in each 
of them. 

The subjects were twenty ‘regular’ students in an Israeli suburban high school, in the 
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last month of their ninth grade school year. They were all volunteers and joined the 
experiment on a first come basis. There was no randomizing in the procedure of 
assigning the volunteers to one of the two groups of subjects, U and D. The 
experiment took place at school, in a room especially assigned for the experiment. 
We took each of the subjects out of class for an individual learning session of 
approximately three hours. 

The module kept track of all the time periods that the subjects spent at each point 
(assignment) as well as of their answers. At the same time, it created a personal file 
for each of the subjects, which was later used to assess the learning process of the 
individual. Moreover, the module enabled the researcher to assess each of the 
answers (to about 204 short questions), according to approximately forty different 
indices. Subsequently, the module created a file of numerical vectors for each subject 
as a product of the evaluation stage, where each of the vectors represented the 
progression of the learning process according to one of the indices. These vectors 
were readable by the computer algebra system, Mathematica, which we used for 
constructing a graphical profile for each subject. The final analysis of the findings of 
our experiment was based primarily on these graphical profiles. 

THE RESEARCH METHODOLOGY 

Students were given the Mathematical Relation (MR) concept to learn because it is 
both highly abstract and difficult to learn. This corresponds with Bereiter’s (1990) 
recommendation to use difficult concepts in order to explore learning processes. The 
MR generalizes many of the concepts that students learn in school: intuitive relations, 
functions, linear transfonnations, mathematical operations, etc. The learning of the 
MR concept in the experiment enabled us to investigate the abstraction capabilities of 
the participants as they tried to relate the new MR concept to more basic concepts 
they already knew. Since our main goal was to concentrate on a thorough, in-depth 
analysis of a short learning episode, we limited the learning to the definition 1 of the 
MR concept. However, in order to avoid having to teach the students such concepts 
as the Cartesian product of two sets, sub-sets etc., we used a simplified version" of 
the formal mathematical definition of the MR. 

Since our interest lies in the learning processes of individuals, we strove to limit the 
impact of the teacher and of other external factors on the learning process. 
Consequently, we used a computerized learning module to convey the learning text to 
the participants. We presented the concept using examples, rather than explanations, 
and presented these examples in a ‘neat’, ‘facts-only’ format without any elaboration. 
This was followed by fixed repeated reasoning assignments to force the learner to 
‘think aloud’. 



1 R is a MR from a set A to a set B if R is subset of the Cartesian product AxB. 

“ R is a MR from a set A to a set B if R is a set of ordered pairs of the form (a,b) where a is 
an element of A and b is an element of B. 
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Basically, our research tool was a computerized module with four functions: 
instigating (a learning module); monitoring (interactive features); quantifying 
(assessment features) and externalizing (graphical profiling) the target processes. 

The Learning Module. In this paper we use the structure of the text as an operational 
definition for the processes studied. Hence, the learning process initiated by a text 
that presents the subject to be learned in an order of increasing abstractness will be 
referred to as an Abstraction Process (AP), while the ‘inverse’ process that is initiated 
by a text that presents the learning subject in an order of decreasing abstractness will 
be referred to as a Reversed Abstraction Process (RAP) 3 . The abstractness level of a 
specific example of the MR was determined by an abstractness scale we devised 
according to the mathematical abstractness level of its different components and its 
different characteristics in widening circles of mathematical abstraction (see also 
Mitchelmore & White, 1995). 

Our learning modules presented fifteen levels of abstraction, which we theoretically 
defined before the experiment. However, in our final analysis we referred only to 
three main levels (low, intermediate and high level of abstractness) and each of these 
levels was again divided into two sub-levels (low and high). 

Each of the fifteen chapters contained positive and negative examples of MR, all of 
approximately the same level of abstractness. In the bottom-up module (group U - 
AP) the chapters are presented in an order of increasing abstractness and in the top- 
down module (group D - RAP) they are in an order of decreasing abstractness. It is 
worth noting that the two modules are identical except for the order in which the 
chapters appear. 

The Monitoring Device. When designing the assignments we aimed towards a 
maximal exposure of the participants' thoughts. Thus, following each of the 
examples, the module presented several kinds of assignments: a) Reasoning 

assignments; b) Construction of examples assignments; c) Counting elements 
assignments; and d) Identification assignments. At the end of each of the fifteen 
chapters the learners were asked to: (a) Describe their feelings, (b) Define a MR 
verbally, and (c) Construct an example of a MR. 

The Assessment Device. Assessment of the learning processes was performed by 
means of thirty-nine indices that were designed to evaluate and record the 
progression of the conception of ten different aspects of the MR and of the three 
general aspects of the learning process (time, affective and fixation aspects). These 
indices were used to evaluate and grade the efforts of individual subjects in each 
point (point=assignment, 204 points in total) of the learning process in an 
accumulated manner, and were normalized to values between -1 and +1 for purposes 



3 Though interesting, yet, the theoretical attributes and significance of the AP and the RAP 
as abstraction processes lie outside of the scope of this paper and therefore will not be 
discuss it here 
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of comparison. To illustrate: if the value of the positive index that measured the 
conception of the Set aspect of the MR had an accumulated value of 0.5 by a certain 
point, this would mean that by this point the student had gained half of what we 
considered to be the ‘full’ positive conception (understanding) of the MR as a Set. 

The assessment was done manually according to the different scales and was then 
typed up. For the most part, there were three indices associated with each of the 
aspects. One measured only the progression in a positive direction (i.e. 
mathematically correct statements and doings, or positive feelings), the second 
measured only the progression in a negative direction (i.e. mathematically incorrect 
statements or doings and negative feelings). The third measured the total progression 
of the conception of the specific aspect - i.e. the sum of both the positive and the 
negative indices. The fixation aspect of the learning process was assessed only by a 
negative index, while the time aspect was assessed by two indices, both positive. One 
measured the time spent at each point and the other measured the elapsed time from 
the beginning of the learning process to a specific point. Three summation indices, 
positive, negative and total, assessed the participants’ total positive, total negative 
and total conception of the MR concept. 

The grading at each point was done in a positive manner; i.e. grades were assigned 
only to the participants’ statements. There was no negative grading for a missing 
statement, or erroneous statements. To illustrate, if a participant neglected to write 
‘ordered pairs’ in his answer regarding an example of MR, it was not documented in 
either the negative or positive indices. On the other hand, if he did have pairs in his 
example, but with curly parentheses rather than round, it was documented in both the 
positive and the negative indices. The positive index was scored for the kind of 
elements (i.e. pairs) and for the existence of parentheses, but not for their shape. At 
the same time, the negative index was scored for the shape of the parentheses. 

The grading was done in an accumulated manner; in the above example, a new 
occurrence of an ordered pair with round parentheses would change the grading of 
the negative index that assessed the conception of the kind-of-elements aspect by 
taking away the points that were assigned previously to the wrong shape of 
parentheses, and at the same time the positive index would be scored for the correct 
shape of parentheses. 

Graphical Profiling. The progression of the indices’ values was graphed on three 
axes: the Process Progression Axis - Point x on this axis marked the X th assignment to 
be handled (one of 204); the Location Axis - shows the location of the point in the 
bottom-up module; the Time Axis - shows the elapsed time from the start. Note that a 
specific point on each of the axes has a different history with regards to the two 
observed processes. The graphs were processed by the Computer Algebra 
Mathematica, based on a file of numerical vectors (204- dimensional) that the 
modules produced for each of the individual subjects. 

The learning processes of each of the twenty individual subjects, ten in each group, U 
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and D, were presented as an individual profile consisting of approximately sixty 
graphs (most of them continuous; few discrete) and a non-graphic set of data. The 
‘average process’ of each of the two groups of learner was also presented via such a 
profile. 

RESEARCH CONCLUSIONS 

We derived dozens of ‘local’ conclusions about the conception of each of the ten 
aspects of the MR and about each of the three general aspects of the learning (time, 
affective and fixation), as well as more ‘global’ conclusions about the whole 
conception process. There were also conclusions concerning the effects of examples 
of a specific level of abstraction on the learning process. 

In the following graphs the dotted line represents the process in group D - the RAP, 
while the continuous line represents the process in group U - the AP. The vertical 
lines represent the different chapters (each chapter presents examples of different 
levels of abstractness) of the bottom-up module. In total the x-axis shows 204 points; 
each represents a short assignment. The y-axis represents the measure of 
effectiveness. 

Figure 1 Conception of the set aspect Figure 2 Total conception 




Figure 1 shows the average progression of the positive conception of the Set aspect of 
the MR concept in both groups of student, U and D. We can see that the average 
achievements of the students in group D were slightly higher than those in group U at 
the end of the learning process, even though along most of the learning process axis, 
group U students did better. This is seen also in the progression of the total 
conception of the Set aspect. This demonstrates the accumulative dominancy of the 
one process over the other, and leads to the conclusion that The RAP is more 
dominant than the AP in respect to the conception of the Set aspect of the MR. 

Figure 2 shows the progression of the total conception of the MR concept in both 
groups of student. We can clearly see that along the learning process axis, the 
achievements of the students in group U (bottom-up), were higher than those in group 
D (taking into account the negative conception as well). This leads to the conclusion 
that the AP is more effective in respect to the total conception (positive minus the 
negative) of the MR concept. This is effectiveness in the sense that at any given 
moment in the learning process, the students in group U gained a greater part of what 
was considered to be the full conception of the MR concept. 

Figure 3 presents the progression of the average negative conception of the Size 
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Figure 3 Negative conception 



aspect of MR in both groups of learners, U and 
D. Here we can clearly see that the graph of 
group D (top-down) is higher than the graph of 
group U. This leads to the conclusion that on 
average, students in group D made fewer errors. 

This in turn implies that the RAP is better at 
leading the conception towards the positive 
direction. We find it intriguing that this is seen in the graphs that depict the 
progression of the conception of most of the aspects of the MR, as well as in the one 
depicting the total conception of the MR concept. 



Figure 4 The Affective aspect 




Figure 5 The Time aspect 




Figures 4 and 5 present the progression of the Affective and the Time aspects of the 
learning process in both groups. From here we can see that there are no significant 
differences between the two processes, AP and RAP, in respect to the Elapsed Time 
aspect nor in respect to the Affective aspect of the learning process. This is especially 
interesting in light of the prevalent view that considers the deductive approach among 
the causes of affective and cognitive difficulties in learning mathematics. 

DISCUSSION 

We argue that our (almost) continuous probing does indeed offer an outline of the 
progression of learning processes. We do, however, make a cautious comment 
regarding the sensitivity of the scales we used to assess each aspect of the monitored 
process, at each point. As we see it from our exploratory study, the scales are the 
weakest link. We nevertheless believe that continual studies, on larger and less biased 
samples, will eventually yield an accurate depiction of learning processes. 

Our study encompasses other weaknesses too: it focuses on a very short learning 
episode (about three hours), while meaningful learning could take much longer (even 
years). It monitors the conception of a very small ‘piece’ of a whole concept (MRs 
could be the subject of years of study) and ‘flattens’ the complex research setting 
(putting a rich learning process into a graphical profile, even if it does contain dozens 
of graphs); it ‘extrapolates’ without sufficient justification (when stretching a line 
between the conception in two adjacent points to achieve the continuousness of the 
process); and it also attempts to derive general conclusions based on a non-statistical 
sample (small and non-random). 

But at the same time, the short learning episode allows us to concentrate on deep and 
detailed observation, and the ‘flat’ graphical portrayal enables us to conduct a precise 
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analysis of the findings. While the design of the modules could have affected the 
processes that we observed, they also facilitated a close monitoring of the learning 
process. And lastly, the simultaneous nature of the tool that allowed us to observe the 
process from as many angles as we desired and had the resources for, gives this kind 
of research enormous potential. 

One of the concerns that prompted the qualitative reform was the segmented nature 
of research on learning up until this point. Qualitative scholars argue quite rightly that 
one needs to see the ‘whole’ learner, which for some includes also the learner’s social 
envelope, in order to study complex learning processes. The tools that qualitative 
scholars advocate are indeed comprehensive and thorough. Yet at the same time they 
fail to make irrefutable general claims about their research findings (Schoenfeld, 
1999). The question is: do we have anything to gain by quantifying a qualitative 
situation? We believe that we do. Moreover, we believe that our methodology, while 
not qualitative, does indeed encompass the essence of qualitative methodologies. We 
believe it offers a holistic, comprehensive and thorough description of the learner - all 
of which are considered qualitative qualities and yet, at the same, also offers the 
means to make irrefutable general claims about the learning process. 

The method we used, in which we probed every minute or so, may be compared to 
the kinds of observation made by qualitative researchers when they closely observe a 
subject. The kinds of question that the learners answered in our research are in many 
ways similar to those asked by qualitative researchers. Taking into account 
simultaneously all the aspects involved in the learning situation is in fact a kind of 
holistic approach. The methodology used could also monitor the physical aspects of 
learning (by incorporating medical devices), the social aspects (by video recording), 
and the motivational aspects (by incorporating psychological tools). Moreover, if one 
argues that the absence of the human perspective might affect the description, we can 
suggest numerous ways in which that perspective might be incorporated as a further 
monitoring device. 
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